Stable Branching Rules for Classical Symmetric Pairs

Given completely reducible representations, V and W of complex algebraic groups G and H respectively, together with an embedding H →֒ G, we let [V,W ] = dimHomH (W,V ) where V is regarded as a representation of H by restriction. If W is irreducible, then [V,W ] is the multiplicity of W in V . This number may of course be infinite if V or W is infinite dimensional. A description of the numbers [V,W ] is referred in the mathematics and physics literature as a branching rule. The context of this paper has its origins in the work of D. Littlewood. In [Li2], Littlewood describes two classical branching rules from a combinatorial perspective (see also [Li1]). Specifically, Littlewood’s results are branching multiplicities for GLn to On and GL2n to Sp2n. These pairs of groups are significant in that they are examples of symmetric pairs. A symmetric pair is a pair of groups (H,G) such that G is a reductive algebraic group and H is the fixed point set of a regular involution defined on G. It follows that H is a closed, reductive algebraic subgroup of G. The goal of this paper is to put the formula into the context of the first named author’s theory of dual reductive pairs. The advantage of this point of view is that it relates branching from one symmetric pair to another and as a consequence Littlewood’s formula may be generalized to all classical symmetric pairs. Littlewood’s result provides an expression for the branching multiplicities in terms of the classical Littlewood-Richardson coefficients (to be defined later) when the highest weight of the representation of the the general linear group lies in a certain stable range. The point of this paper is to show how when the problem of determining branching multiplicities is put in the context of dual pairs, a Littlewood-like formula results for any classical symmetric pair. To be precise, we consider 10 families of symmetric pairs which we group into subsets determined by the embedding of H in G (see Table I in §3).